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Abstract 

Whether  designing  from  scratch  or  modifying  an  existing  command  and  control  (C2)  platform 
one  should  always  take  into  account  the  vast  amount  of  background  research  that  has  been 
performed  in  both  the  psychological  as  well  as  the  graphical  design  literature.  However, 
surveying  this  vast  literature  can  be  daunting  as  well  as  overwhelming  if  one’s  background  is  not 
in  the  applied  science  (e.g.,  psychology).  A  review  is  given  of  the  most  relevant  issues  in  the 
literature  as  well  as  how  the  findings  may  be  applied.  For  example,  for  graphical  design  the  work 
of  authors  such  as  Tufte,  Ware,  and  Healy  is  discussed,  and  how  their  approaches  help  one  to 
better  design  graphical  interfaces.  Likewise,  a  review  of  the  psychological  literature  will 
discusses  issues  such  as  attentional  capacity  issues  (e.g.,  attentional  capture,  change  blindness) 
and  memory  issues  (short  versus  long  tenn  versus  working  memory),  among  others.  Finally,  a 
proposed  method  for  taking  into  account  and  applying  these  principal  for  the  design  of  an 
optimal  C2  system  is  discussed. 

Introduction 

Modern  C2  operations  are  becoming  increasingly  complex.  At  the  same  time,  the  deluge  of 
information  generated  by  the  increasing  use  of  advanced  sensors  and  networks  in  the  battlefield 
exerts  tremendous  informational  and  perceptual  load  on  commanders.  The  massive  amount  of 
data  that  flows  from  different  sources  makes  it  difficult  for  users  to  understand  the  information 
available.  Commanders  may  also  have  difficulty  in  extracting  required  information  from  the 
constant  streams  of  data  resulting  in  critical  data  sources  being  ignored  or  not  well  utilized, 
because  the  visualization  techniques  for  presenting  information  on  current  C2  systems  are 
limited.  The  difficulty  in  understanding  and  exploiting  all  available  information  may  lead  to  a 
reduction  in  the  effectiveness  in  the  conduct  of  military  operations. 

C2  developers  need  to  have  a  deep  understanding  of  how  humans  perceive  and  process 
information  visually,  make  sense  of  information,  and  interact  with  computer  interfaces,  so  as  to 
increase  the  perceptual,  cognitive,  and  information  utility  of  C2  systems.  They  should  take  into 
account  the  vast  amount  of  background  research  and  development  that  has  been  conducted  in 
graphic  design,  perception,  and  cognition  literature. 

This  paper  reviews  the  most  relevant  concepts  in  the  literature  on  graphics  design,  visual 
perception,  cognition  and  information  visualization  evaluation  that  exploit  human  innate 
capabilities  to  enable  rapid  perception  and  understanding  in  the  design  and  development  of  C2 
systems.  The  paper  is  organized  as  follows:  the  first  section  outlines  the  graphic  design 
guidelines  that  exploit  pictographic  and  typographic  elements  to  enhance  visual  communication 
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of  information;  the  second  section  describes  empirically  tested  principles  and  theories  on  how 
humans  perceive  and  organize  visual  infonnation  that  can  be  applied  to  the  design  and 
development  of  C2  related  information  visualization;  the  third  section  relates  issues  of  human 
memory  to  C2  displays;  the  fourth  section  discusses  evaluation  guidelines  and  takes  a  task- 
oriented  approach  to  describe  the  evaluation  of  information  visualization  systems;  finally  the  last 
section  concludes  by  describing  a  proposed  method  for  taking  into  account  and  applying  these 
principles  for  the  design  of  C2  system. 

Graphic  Design 

Users  gain  insight  from  visualizing  the  dataset  when  the  intended  message  of  the  dataset  is 
communicated  effectively  to  the  users.  The  communicative  value  of  the  visualization  can  be 
enhanced  by  graphic  design  which  uses  pictographic  and  typographic  elements  to  create  effective 
visual  communication.  Graphic  design  can  help  to  emphasize  the  important  features  and 
relationships  in  the  dataset  while  minimizing  the  distracting  effects  of  extraneous  details  and  also 
improve  the  presentation  of  the  task  information  through  careful  selection  and  arrangement  of 
graphical  elements  to  establish  clear  visual  relationships  among  the  elements  in  the  composition. 
This  section  presents  a  summary  of  the  concepts  in  graphic  design  layout,  typography,  color,  and 
data  graphics  that  would  serve  as  the  basis  in  designing  the  visualization. 

Layout 

Layout  concerns  how  the  graphical  elements  are  placed  in  the  composition  to  achieve  fast  and 
accurate  comprehension.  The  primary  considerations  of  layout  are  proportion,  format,  and  grid 
system.  The  classical  proportions  used  in  graphic  design  provide  aesthetic  appeal  and  can  be 
used  to  define  the  boundary  of  the  graphical  elements  as  well  as  the  overall  composition.  They 
consist  of  the  square,  square  root  of  2,  golden  rectangle,  and  double  square  (Marcus  1992).  The 
format  refers  to  the  outer  boundary  of  the  composition.  The  grid  divides  the  format  into  sets  of 
horizontal  and  vertical  lines  for  placement  of  graphical  elements.  The  grid  system  is  used  to 
define  key  alignments  of  graphical  elements  within  the  composition  as  well  as  to  establish  a 
visual  hierarchy  of  the  graphical  elements  to  facilitate  the  scanning  of  information.  Decisions  on 
the  placement  of  graphical  elements  can  be  complemented  by  the  use  of  Gestalt  principles  (see 
Section  3.3)  to  assist  the  users  in  perceiving  and  recognizing  patterns  in  information. 

Typography 

Typography  is  the  art  and  technique  of  designing  textual  information  using  a  combination  of 
typeface,  type  size,  line  length,  line  spacing  and  letter  spacing.  The  goal  is  to  optimise  legibility 
and  readability  of  the  textual  context  to  facilitate  interpretation  and  comprehension  of  the 
information.  Typeface  refers  to  the  name  of  the  type  such  as  Times  new  Roman  or  Arial  and  is 
broadly  classified  as  either  serif  or  sans  serif.  San  serif  typefaces  are  generally  recommended  for 
screen  of  low  resolution  or  the  type  is  10  points  or  smaller  (Galitz  2002).  The  details  of  the  serif 
typeface  are  lost  when  the  type  is  small  in  size.  Due  to  the  low  resolution  of  the  screen,  type 
sizes  of  12  and  14  are  recommended  to  provide  comfortable  reading  although  10-point  type  is 
legible  in  small  amounts  (Kahn  and  Lenk  1998).  The  number  of  typefaces  used  in  the 
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visualization  should  be  limited  to  three  in  a  maximum  of  three  sizes  (Marcus  1995)  as  too  many 
typefaces  may  distract  the  users  from  reading  the  infonnation. 

For  continuous  reading  tasks,  mixed  case  text  is  recommended.  The  use  of  upper  case  text  can 
slow  reading  speed  by  12  percent  (Marcus  1995).  For  non-reading  tasks  such  as  identifying 
single  word  or  short  phrases,  perfonnance  is  better  with  upper  case  text  (Tinker  1963).  Upper 
case  text  could  be  used  for  screen  headings  or  cueing  important  items  such  as  warning  (Galitz 
2002).  A  range  of  40-60  characters  per  line  is  a  comfortable  maximum  for  reading  (Marcus 
1995).  In  studies  on  viewer  preferences,  Garbinger  (1984,  1987,  1993)  found  that  readers  prefer 
shorter  lines  of  text  especially  for  single  line  spacing  between  lines.  As  the  line  becomes  longer, 
it  becomes  increasingly  difficult  for  the  eyes  to  sweep  to  the  start  of  the  next  line.  Line  spacing 
of  one  to  one-and-one-half  times  the  type  size  (Galitz  2002)  is  recommended  to  differentiate 
between  lines.  Too  little  line  spacing  will  cause  letters  from  the  line  above  to  join  or  touch  the 
letters  on  the  line  below  and  disrupt  the  flow  of  the  reading. 

Color 

Color  is  a  powerful  visual  element  and  can  enhance  the  communicative  effectiveness  and 
retention  of  the  message  to  the  user.  Perfonnance  can  be  enhanced  considerably  when  color  is 
used  strategically  to  support  task  and  aid  in  decision  making  (Christ  1975).  Conversely,  arbitrary 
use  of  color  can  reduce  the  usability  of  the  visualization.  Color  can  be  described  in  terms  of  three 
dimensions:  hue,  value  or  intensity,  and  saturation  or  chroma.  Hue  is  the  spectral  wavelength 
composition  of  the  color  consisting  namely  red,  orange,  yellow,  green,  blue,  and  violet.  Value  or 
intensity  is  the  relative  amount  of  lightness  or  darkness  of  the  color  which  ranges  from  black  to 
white  (Galitz  2002).  Saturation  or  chroma  is  the  purity  of  the  color  which  ranges  from  grey  to  the 
most  vivid  variant  of  the  color  (Galitz  2002). 

Color  Discrimination 

Color  is  more  effective  than  other  cues  such  as  shape,  size  or  brightness  for  search  tasks  and 
symbol  identification  tasks.  (Christ  1975).  As  the  density  of  symbols  in  the  display  or  the 
number  of  non-target  symbols  increases,  the  perfonnance  advantage  of  color  coding  increases 
(Christ  1975).  The  number  of  colors  should  be  no  more  than  four  to  five  colors  spaced  far  apart 
on  the  color  spectrum  for  tasks  that  require  color  memorization  and  absolute  discrimination 
(correct  identification  of  color  while  no  other  color  is  in  the  field  of  view  (Galitz  2002)). 

Color  Associations 

The  use  of  color  should  conform  to  the  conventions  used  in  the  job  and  the  culture  of  the  users. 
Table  1  shows  some  of  the  colors  that  have  been  commonly  associated  with  particular  meanings 
(Galitz  2002).  Unintended  interpretations  may  arise  when  the  color  conventions  are  assigned 
other  meanings.  Task  perfonnance  may  also  be  affected  as  the  color  conventions  are  ingrained  in 
the  behaviour  of  the  users  and  difficult  to  unlearn.  For  example,  using  other  colors  than  blue  to 
represent  own  units  in  military  displays  may  cause  confusion  in  interpretation  among  the  military 
users. 
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Table  1:  Common  color  associations 


Color 

Associated  meanings 

Color 

Associated  meanings 

Red 

Stop,  fire,  hot,  danger 

Blue 

Cold,  water,  calm,  sky,  neutrality 

Yellow 

Caution,  slow 

White 

Neutrality 

Green 

Go,  OK,  clear,  vegetation,  safe 

Grey 

Neutrality 

Color  Consistency  and  Redundancy 

Color  coding  should  be  applied  consistently  throughout  the  visualization  to  reduce  cognitive 
efforts  and  errors  in  interpreting  the  meanings  of  the  colors  for  different  contexts.  When 
important  information  is  being  communicated,  color  should  be  supported  by  another  redundant 
cue.  This  is  to  facilitate  users  with  anomalous  color  vision  to  identify  the  important  information 
other  than  color  discrimination.  It  has  also  been  shown  that  redundant  coding  enhances 
performance  (Christ  1975). 

Colors  to  Avoid 

Opposing  colors  (yellow  and  blue,  or  red  and  green),  highly  saturated  colors,  and  colors  far  apart 
on  the  color  spectrum  (for  example  red  and  green)  may  cause  afterimages  and  shadows  when 
used  heavily  on  the  same  display  or  across  displays  (Galitz  1985,  ISO  1988).  The  use  of  pure 
blue  should  be  avoided  for  text,  thin  lines  and  small  shapes  (Galitz  2002)  due  to  poor  visual 
acuity  for  shorter  wavelength  of  the  visible  spectrum.  Insensitivity  to  blue  also  increases  with 
age  (Murch  1987).  ISO  (1988)  recommends  avoiding  saturated  blue  for  text  on  dark 
backgrounds. 

Data  Graphics 

Data  or  statistical  graphics  refers  to  the  presentation  of  data  in  graphical  fonn.  Common  types  of 
data  graphics  would  include  bar  graphs,  line  graphs,  scatterplots,  and  pie  charts.  Data  graphics 
can  present  patterns  or  trends  in  data  clearly  and  quickly.  They  also  enable  identification  and 
comparison  of  data  values. 

Data-ink  Ratio 

Data-ink  is  defined  as  the  essential  non-erasable  ink  (pixels)  used  to  present  the  data.  The  goal  is 
to  achieve  the  highest  possible  data-ink  ratio  (Tufte  1983)  without  removing  information 
necessary  for  effective  communication.  This  is  achieved  by  removing  the  non-data-ink  (pixels) 
that  fails  to  present  data  infonnation  and  enhancing  the  data-ink  (pixels).  Figure  1  shows  a 
revised  graph  (on  the  right)  after  erasing  half  the  frame  (remove  non-data-ink)  and  increasing  the 
data  curve  (enhance  the  data-ink)  (Tufte,  1983). 
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Moire  Effects 


Moire  effects  occurs  when  the  users  view  a  set  of  lines  or  dots  that  is  superimposed  on  another 
set  of  lines  or  dots,  where  the  sets  differ  in  relative  size,  angle,  or  spacing.  This  produces 
distracting  appearance  of  movement  and  should  be  avoided  (Tufte  1983). 

Grid  and  Rules  in  Tables 

The  grid  should  be  muted  relative  to  the  data  so  that  its  presence  does  not  compete  with  the  data. 
This  is  achieved  by:  (1)  making  the  grid  thinner;  (2)  making  the  grid  dotted;  and,  (3)  using  a  grey 
grid.  Dark  grid  lines  should  not  be  used  as  they  interfere  the  reading  of  the  graph  and 
interpolating  the  data  (Tufte  1990).  In  creating  tables  the  use  of  rules  should  be  avoided  unless 
they  are  absolutely  necessary.  Vertical  rules  are  needed  only  when  the  space  between  columns  is 
so  narrow  that  mistakes  will  occur  in  reading  without  rules.  Thin  rule  should  be  used  rather  than 
thick  rules  (Tufte  1990). 

Small  Multiples  Design 

Small  multiples  refers  to  information  slices  that  repeat  a  common  design  several  times  within  a 
user’s  eye  span — with  each  instance  showing  different  data  values  (Tufte  1990).  Their  multiplied 
smallness  allows  local  comparisons  at  a  glance  and  the  consistent  design  facilitates  the  users  to 
compare  and  contrast  the  change  in  data.  In  Figure  2,  each  multiple  on  the  lower  portion  of  the 
figure  maintains  a  consistent  frame  of  reference — size,  color,  fonts — with  changing  data — cloud 
shape,  number  of  minutes.  This  provides  a  visual  appreciation  of  how  the  stonn  (or  information 
displayed)  changed  over  time. 


Figure  2:  Small  multiples  showing  change  of  shape  of  the  storm  (Baker  &  Bushell,  1995) 

Number  of  Colors 

The  number  of  colors  for  data  graphics  should  not  be  more  than  six  (Galitz  2002)  to  minimise 
confusion  and  distraction  in  looking  at  the  visualization.  Variables  should  be  encoded  in  different 
hues  for  tasks  that  require  efficient  discrimination  of  colors  (Cleveland  1994). 
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Effectiveness  of  Visual  Variables 


Bertin  (1983)  provided  a  comprehensive  survey  of  mapping  the  graphical  data  representation 
(components)  into  visual  variables  (position,  size,  value,  texture,  color,  orientation,  and  shape) 
and  the  rules  governing  their  effective  use.  The  type  of  data  to  be  represented  in  information 
visualization  can  be  classified  into  the  ordinal,  nominal,  and  quantitative. 

Cleveland  and  McGill  (1984)  broaden  Bertin’s  work  by  identifying  several  other  visual  variables 
and  ranked  them  according  to  different  effectiveness  for  quantitative  data.  MacKinlay  (1986) 
extended  Cleveland  and  McGill's  ranking  to  include  ordinal  and  nominal  data  as  shown  in  Table 
2.  Although  MacKinlay's  ranking  is  not  empirically  verified,  it  provides  a  guide  to  selection  of 
visual  variables  for  effective  representation  of  different  data  types  in  information  visualization. 


Table  2:  Ranking  of  visual  variables  for  different  data  types  (MacKinlay,  1986).  The  variables  in  the 
grey  boxes  are  not  applicable  for  that  data  type. 


Qualitative 

Position 
Length 
Angle 
Slope 
Area 
Volume 
Density 
Color  Saturation 
Color  Hue 
Texture 
Connection 
Containment 
Shape 


Ordinal 

Position  — 
Density 
olor  Saturation 
Color  Hue 
Texture 
Connection 
Containment 
Length 
Angle 
Slope 
Area 
Volume 
—  Shape 


Nominal 

Position 

Color  Hue 

Texture 

Connection 

Containment 

Density 

Color  Saturation 

Shape 

Length 

Angle 

Slope 

Area 

Volume 


Visual  Perception 

Norman  (1993)  discussed  several  basic  principles  for  the  development  of  effective  visual 
representations  which  we  discuss  next.  First,  the  Naturalness  Principle  states  that  experiential 
cognition  is  most  effective  when  the  properties  of  the  visual  representation  map  closely  against 
the  properties  of  the  infonnation  being  represented.  This  principle  suggests  that  artificial  visual 
representations  that  vastly  differ  from  the  infonnation  being  represented  can  actually  have  a 
negative  effect  in  perceptual  understanding. 

The  Perceptual  Principle  states  that  perceptual  and  spatial  representations  are  preferred  over  non- 
perceptual,  non-spatial  representations,  but  only  if  the  Naturalness  Principle  is  followed.  This 
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principle  suggests  that  visual  representations  that  require  reflection  are  not  as  rapidly  understood 
and  perceived  as  those  that  can  be  used  experientially,  through  simple  graphical  perceptual 
comparisons. 

The  Appropriateness  Principle  states  that  visual  representations  should  provide  just  sufficient 
information  that  is  required  for  the  task.  This  principle  suggests  that  additional  infonnation  not 
related  to  the  task  at  hand  should  be  avoided,  as  it  may  actually  cause  distractions. 

Tversky  et.  al.  (2002)  also  discussed  two  basic  principles:  Principle  of  Congruence  and  Principle 
of  Apprehension.  The  Principle  of  Congruence  states  that  the  structure  and  content  of  the 
external  representation  should  correspond  to  the  desired  structure  and  content  of  the  internal 
representation.  This  principle  suggests  that  visual  representations  should  match  and  represent  the 
information  presented.  The  second  principle,  the  Principle  of  Apprehension,  states  that  the 
structure  and  content  of  the  external  representation  should  be  readily  and  accurately  perceived 
and  comprehended. 

In  addition  to  these  aforementioned  basic  principles,  visual  perceptual  theories  have  also 
suggested  how  the  human  visual  system  perceives  structures  and  groups  of  information  from  the 
scene.  By  exploiting  the  visual  perception  capabilities  of  the  human  it  is  possible  to  map  data  to 
visual  representations  that  will  improve  visual  tasks  such  as  pattern  matching,  trend 
identification,  recognizing  gaps  and  error  discovery. 

Extending  these  basic  principles  and  theories,  the  following  sub  sections  will  discuss  and  list  out 
guidelines  pertaining  to  pre-attentive  vision  processing,  attentional  cueing,  gestalt  principles  of 
organization  and  depth  perception  theory  that  can  be  considered  for  information  visualization 
design. 

Pre-attentive  Visual  Processing 

Research  in  vision  and  psychology  has  discovered  that  the  human  low  level  visual  system  rapidly 
processes  information  in  parallel  to  extract  basic  visual  features  of  objects  in  a  scene.  These  are 
simple  shapes  and  colors  that  “pop  out”  from  their  surroundings.  The  theoretical  mechanism 
underlying  the  pop-out  phenomenon  is  called  pre-attentive  processing  as  it  occurs  prior  to 
conscious  attention.  The  visual  features  that  are  pre-attentively  processed  can  be  organized  into  a 
number  of  categories  based  on  form,  color,  motion,  and  spatial  position.  A  survey  of  these 
features  can  be  found  in  (Healey  and  Enns  1999). 

Experimental  studies  in  the  development  of  scientific  visualizations  have  indicated  that  such 
features  can  be  mapped  to  critical  information  to  support  visual  cognitive  tasks  such  as  target 
detection,  boundary  detection,  and  region  tracking  at  a  single  glance  for  rapid  perception  (Healey 
1993).  Appendix  A  describes  empirically  tested  guidelines  that  explain  how  such  pre-attentive 
features  can  be  used  for  designing  and  developing  advanced  visualizations  techniques  for 
supporting  the  above  mentioned  visual  cognitive  tasks. 


Attentional  Cueing 


If  we  say  we  are  going  to  “cue”  attention,  we  must  begin  by  defining  visual  attention,  and  how  it 
works.  When  we  say  that  one  is  attending  to  a  location,  we  are  talking  about  selective  attention. 
That  is,  selectively  filtering  out  unnecessary  information  in  order  to  make  more  efficient  use  of 
the  vast  array  of  available  information.  Selective  attention  research  has  a  very  long  history, 
dating  all  the  way  back  to  James  (1890),  who  some  consider  a  founding  father  of  experimental 
psychology. 

One  may  then  ask  what  are  you  studying  when  you  say  “attention”  and  “selective  attention”? 
Posner  (1990)  answers  this  with:  “Attention  involves  selection  of  higher  levels  of  processing, 
including  conscious  processing,  while  preventing  access  of  other  signals  to  those  same  high 
levels  of  processing.  Selective  attention  plays  an  important  role  in  most  cognitive  tasks  including 
pattern  recognition,  reading,  and  mental  imagery.” 

More  recently,  theories  of  visual  attention  have  fallen  into  two  distinct  camps,  space-based  and 
object-based  theories.  Logan  (1996)  outlined  these  two  theories  as  follows.  Space-based  theories 
argue  that  attention  is  distributed  over  visual  space,  irrespective  of  the  objects  (stimuli)  in  this 
space.  These  theories  often  assume  that  attention  is  like  a  spotlight  moving  around  in  visual 
space.  When  an  object  is  "in"  the  spotlight,  it  is  attended  to,  if  not  it  is  ignored  (Eriksen  & 
Eriksen,  1974;  Eriksen  &  St.  James,  1986;  Posner,  1980;  Posner  &  Cohen,  1984;  Treisman  & 
Gelade,  1980;  Treisman  &  Gonnican,  1988). 

On  the  other  hand,  object-based  theories  assume  that  objects  are  selected,  rather  than  areas  of 
visual  space  (Kahneman  &  Henik,  1981;  Kahneman  &  Treisman,  1984;  Kahneman,  Treisman,  & 
Gibbs,  1992;  Pylyshyn  &  Storm,  1988).  These  theories  assume  that  representations  of  visual 
space  are  organized  according  to  the  Gestalt  laws  of  perception.  In  other  words,  objects  are 
attended  to  as  a  result  the  Gestalt  laws  “focusing”  attention.  For  example,  two  objects  that  are 
near  each  other  may  be  attended  to  due  to  their  proximity.  On  the  other  hand,  objects  that  are  not 
spatially  adjacent  may  be  attended  to  quite  well  if  they  are  similar. 

As  can  be  seen,  there  is  abundant  research  supporting  both  theories.  However,  Logan’s  (1996) 
model  of  visual  attention  accounts  for  both  theories  implying  that  perhaps  both  approaches  are 
used  by  the  brain  depending  upon  the  situation.  Perhaps  more  pertinent  to  this  discussion  is  the 
fact  that  both  theories  allow  us  to  understand  more  completely  how  visual  attention  functions. 

However,  in  a  C2  display  we  are  more  interested  in  cueing  attention  to  a  certain  location  in 
visual  space.  In  this  case,  the  dichotomy  lies  between  endogenous  and  exogenous  visual  attention 
(Briand  &  Klein,  1987;  Posner,  1980).  Specifically,  exogenous  attention  is  an  automatic  response 
to  an  external  cue.  A  typical  exogenous  cue  would  be  a  bar  near  a  peripheral  target.  For  example, 
Henderson  (1991)  used  a  bar  that  underlined  a  target  to  be  reported,  while  Murphy  &  Eriksen 
(1987)  used  a  bar  that  pointed  at  a  target. 

Endogenous  visual  attention,  on  the  other  hand,  is  under  more  voluntary  control  and  the  cues 
used  in  this  case  refer  to  locations  that  are  predicted  to  contain  a  target.  A  typical  endogenous 
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cue  would  be  a  line  or  arrow  at  fixation  that  points  to  where  attention  should  be  directed  (e.g., 
McConnick  &  Klein,  1990;  Posner,  et  al,  1980). 

Perceptual  Organization 

Gestalt  psychologists  were  intrigued  by  the  way  the  human  mind  perceives  wholeness  out  of 
incomplete  elements  (Behrens  1984;  Mullet  and  Sano  1995)  and  proposed  a  theory  of  pattern 
perception  that  relies  on  the  overall  form  and  is  not  predictable  by  considering  the  sum  of  its 
components.  Factors  that  impact  on  the  perception  of  form  and  how  parts  are  grouped  into 
structural  forms  are  captured  in  what  are  called  the  "Gestalt  Principles  of  Organization”.  Gestalt 
principles  describe  how  elements  presented  together  tend  to  be  grouped  into  distinct  patterns.  In 
essence,  Gestalt  principles  can  be  applied  as  abstract  infonnation  visualization  design  guidelines 
for  tasks  showing  relationships  that  describe  how  infonnation  should  be  organized  or  grouped  so 
that  critical  structures  and  relationships  can  be  easily  perceived.  By  mapping  the  information 
structures  to  easily  perceived  patterns,  it  is  easier  to  interpret  the  relationships  as  well  as  the 
structures  of  the  infonnation.  The  information  structures  will  also  be  easily  interpreted  when  they 
are  mapped  to  readily  perceived  patterns.  There  are  in  general  9  Gestalt  principles  of 
organization  that  can  used  as  guidelines  for  the  design  of  information  visualization  which  are 
shown  in  Appendix  B. 

Depth  and  3D  Perception 

Depth  perception  is  the  visual  ability  to  perceive  the  world  in  3D  and  gives  humans  the  ability  to 
gauge  the  distance  to  an  object.  There  are  3  categories  of  depth  cues  being  researched  in  the 
domain  of  depth  perception:  monocular  static  cues,  monocular  dynamic  cues,  and  binocular  cues. 
Monocular  cues  are  produced  by  the  input  of  one  eye,  while  binocular  cues  are  produced  by  the 
input  from  both  eyes.  Appendix  C  describes  guidelines  that  explain  how  depth  cues  can  be  used 
for  designing  and  developing  advanced  visualizations  techniques. 

Human  Memory 

In  this  section  we  discuss  the  cognitive  factors  that  come  into  play  when  designing  for  C2 
information  displays  in  terms  of  human  memory.  One  might  easily  assume  that  interacting  with  a 
display  assumes  no  need  for  memory  but  as  we  will  see  that  is  not  always  the  case.  In  general 
memories  are  first  encoded  (processed  by  the  senses  and  sent  to  the  brain),  then  stored  (through 
various  physiological  processes  in  the  brain),  and  finally  retrieved.  In  terms  of  this  discussion  we 
will  only  elaborate  on  encoding  and  retrieval  as  they  are  most  gennane  to  the  discussion. 

Sensory  Memory 

Memories  can  be  encoded  into  three  different  types  of  memory;  sensory,  short  term,  and  long 
tenn  memory.  We  will  discuss  each  next  in  turn.  Sensory  memory  takes  place  and  lasts  on  the 
order  of  seconds  and  the  classic  example  is  Sperling’s  (1960)  partial  report  technique.  In  his 
experiments  participants  were  shown  three  rows  of  four  letters  each  for  a  brief  time  (e.g.,  150 
ms.).  If  they  were  told  to  report  all  the  letters  they  performed  poorly  and  reported  only  4-5 
letters.  However,  if  they  were  given  an  auditory  tone  (high,  medium,  or  low)  corresponding  to 
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the  row  to  report  they  then  reported  on  average  three  of  the  four  letters.  This  showed  that  the 
whole  array  must  have  been  in  a  sensory  store  and  the  letters  were  all  briefly  available  to  be 
reported  as  the  participants  did  not  know  ahead  of  time  which  row  they  were  to  report. 

Short  Term  Memory 

Short  term  memory  is  less  fleeting  and  is  most  well  known  from  Miller’s  (1956)  seminal  paper 
that  showed  participants  were  able  to  remember  7  plus  or  minus  2  chunks  of  information.  For 
example,  if  a  list  of  ten  numbers  was  read,  a  participant  on  average  would  be  able  to  recall 
between  five  and  nine  numbers.  The  term  chunk  refers  to  the  fact  that  if  pieces  of  infonnation  are 
put  together  into  a  meaningful  whole,  then  more  can  be  recalled.  For  example,  most  people  can 
easily  remember  five;  if  not  more,  telephone  numbers,  which  with  area  code,  five  phone  numbers 
is  a  combined  total  of  50  numbers  recalled. 

Baddeley  and  Hitch  (1974)  proposed  a  model  that  extended  the  idea  of  how  short  term  memories 
are  encoded  and  processed  with  their  working  memory  model  in  which  it  has  been  shown  that 
participants  can  use  either  a  “verbal  store”  (rote  rehearsal)  or  a  “visual-spatial  sketchpad”  (used 
for  visual  memories).  Others  have  extended  the  idea  of  visual  memories  (see  Kosslyn,  1980) 
further  showing  how  non-verbal  memories  are  encoded. 

Long  Term  Memory 

Long  term  memory  is  that  memory  store  that  holds  information  for  anywhere  from  hours  to 
years.  Anderson  (1976)  explains  that  long  term  memory  is  broken  down  into  two  distinct  types 
tenned  declarative  and  procedural.  Declarative  memories  are  those  that  take  into  account  facts 
(semantic)  and  personal  memories  (episodic).  Procedural  on  the  other  hand  takes  into  account 
procedures  we  learn  (e.g.,  motors  skills  such  as  driving  a  car).  The  main  difference  between 
declarative  and  procedural  is  that  procedural  memories  are  termed  implicit  memory,  meaning 
that  no  conscious  recall  of  the  information  is  needed;  unlike  declarative  memories  in  which  one 
has  to  recall  the  information  (sometimes  with  great  effort). 

Relations  to  C2  Information  Display  Design 

In  general,  most  of  the  infonnation  given  to  a  user  would  fall  under  sensory  infonnation  as  all 
they  need  to  do  is  encode  and  act  upon  the  information.  However,  memory  does  interact  to  a 
large  extent  with  previously  discussed  attentional  aspects  of  perception  and  more  recently  several 
papers  have  discussed  the  relation  of  attention  to  working  memory  (Cowan,  et  al,  2005;  Cowan 
and  Morey,  2006).  ft  seems  that  attention  helps  in  the  encoding  of  working  memory  and  the  two 
may  be  intimately  related. 

A  second  aspect  of  memory  that  has  been  given  prominence  lately  is  that  of  prospective  memory. 
The  term  was  first  used  by  Meacham  and  Singer  (1977)  and  describes  the  act  of  remembering  to 
do  something  in  the  future.  For  example,  an  air  traffic  controller  puts  several  airplanes  into  a 
holding  pattern  due  to  weather  and  an  already  increasing  cue  of  prior  traffic  still  waiting  to  land. 
Prospective  memory  studies  how  the  controller  must  remember  to  eventually  take  those  planes 
out  of  the  holding  pattern.  In  terms  of  C2  displays,  one  can  easily  see  the  relevance  of 
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prospective  memory  especially  if  one  is  building  a  tactical  plan  and  needs  to  wait  until  event  A 
occurs  before  event  B  can  be  accomplished.  Similarly,  if  one  has  several  tasks  to  perform  using  a 
display,  one  of  them  may  be  to  refer  back  to  a  certain  area,  troop  movement,  etc.,  and  react  if  a 
certain  event  occurs. 

Perception  Based  Evaluation  Techniques 

The  information  mapped  by  visualizations  must  go  through  the  human  perceptual  system,  thus 
careful  attention  during  evaluation  to  the  perceptual  system’s  characteristics  can  greatly  improve 
the  effectiveness  of  visualizations  (Rushmeier,  et  al,  1997).  The  evaluation  of  the  effectiveness 
of  visualizations  can  substantiate  claims  made  about  the  value  of  a  new  technique,  and  enables 
the  performance  of  comparisons  with  current  established  systems.  This  section  adopts  a  task- 
oriented  pipelined  approach  to  describing  the  evaluation  methods. 

Situation  Awareness 

One  of  the  main  objectives  of  information  visualizations  is  to  enable  a  user  to  perceive  and 
comprehend  the  environment  and  the  dynamics  behind  it,  of  the  past,  present,  and  future  such 
that  action  can  be  taken.  This  is  situation  awareness  (SA),  a  mental  representation  of  the  current 
state  of  a  dynamic  environment.  Endsley  (2000)  characterizes  situational  awareness  into  three 
levels:  Perception,  Comprehension,  and  Projection,  which  captures  the  essence  of  SA. 

In  the  context  of  C2  systems  for  military  applications,  situation  awareness  enables  a  commander 
to  Perceive  and  Comprehend  enemy  troop  movement,  deployment,  fonnation  (i.e.  unit  type: 
artillery,  armour,  infantry,  etc)  and  concentrations  such  as  to  possibly  predict  (Projection)  enemy 
tactics,  and  to  hence  plan  his  own  tactics  to  pre-empt  the  enemy,  and  is  thus  an  important 
subjective  measure  in  an  evaluation  of  how  well  an  information  visualization  performs. 

A  visual  task  based  evaluation  framework 

The  evaluation  of  an  information  visualization  system  (I VS)  is  essentially  the  evaluation  of  how 
well  the  user  performs  tasks  in  the  system  (Hollands,  2002;  Juarez,  2000).  As  such,  this  paper 
adopts  a  model-based  evaluation  framework,  which  models  the  visual  tasks  that  a  user  performs. 

Some  common  evaluation  techniques  which  apply  to  infonnation  visualization  are  the  checklist  - 
which  usually  has  a  limited  response  outputs  of  accept/reject,  questionnaires/interviews  -  which 
incorporate  some  form  of  subjective  valuing,  observational  techniques  -  which  measure 
behaviour  directly  and  thus  are  limited  to  only  behavioural  variables  and  finally  experimental 
techniques  -  which  is  by  far  the  most  common  and  rigorous  means  of  scientific  evaluation. 
These  techniques  are  ah  applicable  to  the  proposed  visual  task  based  evaluation  framework,  and 
depend  on  the  extent  of  evaluation  required  by  the  evaluator  as  is  balanced  by  resources 
available  and  requirements. 

The  following  two  sub-sections  delineate  the  possible  visual  tasks  that  users  may  perform  on 
both  a  low  and  high  level  of  human  perception.  They  serve  as  a  guide  to  what  users  may  be 
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tasked  to  perform  in  order  to  apply  the  above  evaluation  techniques  to  quantify  and  qualify  the 
output  responses  of  the  users. 

Low  level  visual  tasks 

Wehrend  and  Lewis  (1990)  describe  a  low  level,  domain-independent  taxonomy  of  tasks  that 
users  could  perform  in  a  visual  environment.  Their  taxonomy  is  useful  in  suggesting  possible 
tasks  that  could  be  used  for  evaluating  different  pre-attentive  components  in  the  system.  Possible 
output  data  measured  may  include  time  or  accuracy  to  locate  the  infonnation.  The  tasks  are 
described  below. 

•  Locate 

Searching  for  semantic  units  is  one  of  the  most  rudimentary  tasks  required  for 
interaction  in  an  I  VS. 

•  Identify 

The  Identify  task  is  similar  to  Locate,  with  the  main  difference  being  that  the  semantic 
unit  may  not  be  one  that  the  user  has  prior  knowledge  of. 

•  Distinguish 

Semantic  units  can  sometimes  be  designed  with  similar  graphical  layouts  or  values. 
This  may  be  for  the  purposes  of  identifying  that  they  may  or  may  not  belong  to  a 
common  group. 

•  Categorize 

Users  can  be  tasked  to  arrange/organize  semantic  units  such  as  to  follow  particular 
characteristics  such  as  shape  or  color  such  as  to  exploit  human  spatial  memory. 

•  Cluster 

Users  can  be  tasked  to  move  semantic  units  together  into  groups.  This  is  different  from 
a  categorize  task  as  grouping  need  not  depend  on  similar  characteristics. 

•  Distribution 

Users  can  be  tasked  to  separate  semantic  units  into  various  areas  of  the  display. 

•  Rank 

Users  can  be  tasked  to  rank  semantic  units  according  to  a  set  of  conditions  (e.g., 
population  density,  death  rates),  and  is  only  applicable  to  scalar  and  ordinal  data. 

•  Compare  within  entities 

Users  are  tasked  to  determine  an  outcome  based  on  the  attributes  of  similar  objects. 

•  Compare  between  relations 

Comparing  between  relations  refers  to  the  task  of  comparing  attributes  between 
different  objects.  It  is  an  inferential  task  to  derive  both  relational  and  non-relational 
aspects  of  the  attributes. 

•  Associate 

Association  calls  upon  the  user  to  detennine  which  objects  in  the  display  are  related 
and  to  annotate  as  such.  This  is  different  with  the  Compare  between  relations  task  as  it 
is  not  necessary  to  detennine  an  outcome  of  the  relation. 

•  Correlate 

A  correlation  task  detennines  how  well  the  user  is  able  to  discern  objects  which  share 
attributes  with  other  objects. 
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High  level  visual  tasks 


Knowing  the  fundamental  interaction  tasks  is  insufficient  to  describe  the  means  by  which 
humans  perceive  and  cognize.  Zhou  and  Feiner  (1998)  propose  a  visual  task  taxonomy  that  is 
based  on  cognitive  psychologists’  extensive  studies  (Treisman,  1982;  Levie,  1987)  to  understand 
human  visual  perceptual  behaviour,  as  well  as  Wehrend  and  Lewis’s  (1990)  work  that  provides  a 
good  visual  task  framework  to  evaluate  information  visualizations. 

Zhou  and  Feiner  (1998)  characterize  visual  tasks  into  visual  accomplishment  and  visual 
implications: 

“Visual  accomplishments  describe  the  type  of  presentation  intents  that  a  visual  task  might 
help  to  achieve,  while  visual  implications  specify  a  particular  type  of  visual  action  that  a 
visual  task  may  carry  out”  (ibid). 

Visual  task  classification  through  visual  accomplishment  is  inadequate.  This  is  because  visual 
accomplishment  is  domain  dependent,  and  relies  on  the  intent  of  the  presentation  and  the  visual 
tasks.  Take  for  example,  the  blueprints  of  a  house  as  presented  to  an  architect  and  a  structural 
engineer;  while  the  intent  of  the  presentation  of  the  blueprints  to  the  architect  is  of  aesthetics  and 
ergonomics,  the  intent  to  the  structural  engineer  is  for  stability  and  strength  of  materials  used. 

Visual  implication  on  the  other  hand  is  domain  independent,  since  it  does  not  take  into  account 
the  presentation  intent  but  rather,  the  visual  actions  that  each  task  may  carry  out.  Zhou  and 
Feiner  formulated  3  types  of  visual  perception  and  cognitive  principles:  visual  organization1 2 , 
signalling '  and  transformation3 . 

Table  3:  Zhou  and  Feiner’s  (1998)  visual  implication  framework  as  presented  in  (Morse,  Lewis,  and  Olsen 
_ 2000) _ 


Visual  Implication 

Type 

Subtype 

Elemental  tasks 

Organization 

Visual  grouping 

Proximity 

Associate,  cluster,  locate 

Similarity 

Categorize,  cluster,  distinguish 

Continuity 

Associate,  locate,  reveal 

Closure 

Cluster,  locate,  outline 

Visual  attention 

Cluster,  distinguish,  emphasize,  locate 

Visual  sequence 

Emphasize,  identify,  rank 

Visual  composition 

Associate,  correlate,  identify,  reveal 

Signalling 

Structuring 

Tabulate,  plot,  structure,  trace,  map 

Encoding 

Label,  symbolize,  portray,  quantify 

Transformation 

Modification 

Emphasize,  generalize,  reveal 

Transition 

Switch 

From  Table  3,  this  high  level  visual  task  framework  provides  a  better  model  of  how  the  human 
perceptual  and  cognitive  systems  work  than  the  elemental  tasks  given  above.  This  high  level 
framework  gives  a  better  fit  for  the  proposed  evaluation  methodology. 

1  Visual  organization  suggests  how  people  organize  the  world  and  perceive  it  as  a  coherent  whole. 

2  Visual  signalling  explains  how  people  tend  to  interpret  visual  cues  and  infer  their  meanings. 

3  Visual  transformation  explains  how  people  switch  attention  and  adapt  to  visual  changes. 
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Evaluation  can  now  take  the  form  of  questioning  (through  the  application  of  the  evaluation 
techniques)  how  well  visual  grouping,  visual  attention,  etc.  is  enabled  in  the  I  VS,  which  subtend 
the  elemental  tasks.  This  taxonomy  thus  provides  a  comprehensive  and  complete  evaluation  of 
information  visualizations. 

How  to  Apply  to  C2  Information  Displays 

The  question  that  arises  from  this  entire  discussion  is:  “How  does  one  apply  this  to  the  design  of 
a  C2  display?”  We  argue  that  one  need  not  apply  the  entire  set  of  guidelines  en  masse  but  rather 
they  should  be  used  more  as  a  reference  to  check  the  ideas  that  are  put  forward  in  the  design  of 
an  information  display.  For  example,  one  could  refer  to  the  section  on  Data  Graphics  when 
designing  small  informational  graphics  to  be  used  on  a  display.  Here  one  can  get  guidelines  for 
the  best  way  to  design  data  graphics  as  well  as  find  references  if  more  information  is  needed. 

Likewise  an  already  existing  information  display  could  be  evaluated  in  terms  of  the  guidelines 
set  forth  here  and  used  as  a  checklist.  One  example  might  be  an  evaluation  of  the  attentional 
cueing  components  designed  into  a  display.  Here  one  could  reference  back  to  the  section  on 
Attentional  Cueing  and  evaluate  if  the  techniques  being  used  are  efficient  and  will  indeed  cue 
attention  as  needed.  For  example,  a  review  of  endogenous  and  exogenous  cues  and  their 
definition  may  help  one  design,  or  modify  the  design,  of  an  existing  cueing  system. 

What  if  one  wants  to  build  a  system  from  the  ground  up?  How  would  these  guidelines  be  used 
then?  In  this  case  we  would  argue  three  points.  First,  no  one  truly  builds  a  system  from  the 
ground  up  without  any  prior  knowledge  of  a  similar  system.  Further,  any  similar  system  will  be 
known  to  have  good  and  bad  features  of  which  these  will  be  taken  into  account  in  the  design  of 
the  new  system.  Second,  when  building  a  system  from  the  ground  up  there  will  be  separate 
components  that  may  be  tested  by  themselves  before  being  added  into  the  system  as  a  whole. 
Finally,  this  set  of  guidelines  is  not  intended  to  be  an  exhaustive  list  of  all  issues  dealing  with 
information  displays.  In  fact,  a  well  coordinated  team  will  have  a  good  mix  of  multidisciplinary 
backgrounds  so  that  their  foundations  may  be  incorporated  and  lastly  one  must  somehow  test  the 
system  before  fielding. 

This  leads  to  the  final  question:  “How  does  one  test  a  visual  information  display?”  While  this  is 
quite  an  open-ended  question  we  suggest  the  following  points.  First,  as  previously  mentioned  it 
is  much  easier  to  test  individual  components  to  assess  their  utility  and  fine-tune  their  workings 
before  putting  them  into  the  system  as  a  whole.  For  example,  one  can  easily  test  attentional  cues 
in  a  simple  environment,  changing  them  as  necessary  to  work  in  the  operational  environment. 
Secondly,  testing  components  allows  one  to  use  naive  participants  who  do  not  need  to 
understand  the  final  product  but  can  be  used  to  test  important  factors  such  as  legibility,  color 
recognition,  etc.,  before  the  system  is  shown  to  the  end-user.  In  fact,  the  final  product  may  be 
tested  with  naive  participants  to  get  an  understanding  of  potential  training  requirements  as  they 
(the  naive  users)  have  no  preconceived  idea  of  how,  what,  when,  where,  or  why  things  are 
supposed  to  be  done.  Finally,  the  last  step  would  be  to  test  in  a  semi-or  full-operational 
environment  with  actual  end-users.  In  this  case  one  is  less  likely  to  be  perfonning  specific 
empirical  experiments,  and  more  likely  to  be  observing  the  user  and  understanding  their  likes 
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and  dislikes  with  the  system.  Objective  measures  can  be  recorded,  but  often  one  is  not  able  to  run 
enough  trials  to  be  able  to  perfonn  a  statistical  analysis  with  out  referring  to  little-known  (but 
acceptable)  statistics  for  small-n  designs  (for  an  excellent  review  of  statistics  for  single-case  and 
small-n  experimental  designs  see  Todman  and  Dugard,  2001). 

Overall,  an  attempt  has  been  made  to  present  a  set  of  overarching  guidelines  for  use  in  the  design 
of  a  C2  infonnation  display.  Several  issues  have  been  outlined  ranging  from  graphical  design  to 
experimental  psychology  and  suggested  ways  in  which  these  guidelines  should  be  used  were 
given.  Finally,  issues  dealing  with  how  one  might  be  able  to  test  a  C2  information  display  were 
discussed. 
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Appendix  A:  Empirically  tested  guidelines  on  pre-attentive  features  for 
designing  and  developing  advanced  visualizations  techniques. 


Pre-attentive 

Task 


Pre-attentive 

Feature 


To  identify 
form,  shape 
and  detailed 
patterns  of 
visual  objects 


Color 


Flicker 


Velocity 


Guidelines 

Use  a  sequence  with  a  substantial  luminance  component  (i.e.  black  and  white) 
in  order  to  assist  the  individual  to  interpret  detailed  information  of  the  visual 
object. 

Use  different  levels  of  saturation  to  encode  different  categories  of  visual 
object. 

Do  not  use  more  than  5  colors.  Avoid  hues  like  turquoise  and  lime  green,  as 
they  are  perceived  more  ambiguously  (Kosara,  Healey,  Interrrante,  Laidlaw 
and  Ware  2003). _ 

Flicker  must  be  coherent.  If  there  are  more  than  two  flickering  objects,  they 
must  flicker  at  the  same  rate.  The  cycle  length  has  to  be  120  ms  or  greater  for 
preattentive  detection  (Huber  and  Healey  2005). _ 

Higher  initial  velocity  produces  a  faster  response  to  changes  in  velocity.  If  an 
object  has  a  velocity  of  100  pixels  per  second  (pps),  the  detection  of  a  change 
to  10  pps  is  much  faster  than  an  object  with  initial  velocity  of  20  pps.  When 
the  target  velocity  is  double  of  its  initial  velocity,  detection  is  preattentive  (~50 
ms)  (Hohnsbein  and  Mateeff  1998). 


To  detect  a 
target 


There  must  be  a  minimum  of  16  pps  change  difference  between  initial  and 
current  velocity  for  the  viewer  to  detect  a  change  preattentively  (Huber  and 
_ Healey  2005). _ 

Direction  of  Direction  of  target  object  must  differ  by  at  least  20°  to  be  detected  (Huber  and 
Motion _ Healey  2005). _ 

Height  and  density  can  be  used  to  form  texture  patterns  that  can  be  identified 
preattentively  as  shown  in  Figure  3.  However,  regularity  can  only  be  used  as  a 
secondary  dimension.  It  is  recommended  to  use  height  to  represent  primary 
dimension,  as  height  can  be  identified  at  preattentive  exposure  durations  (150 
ms  or  less)  with  very  high  accuracy  ( — 93%).  This  is  independent  of 
background  density  and  regularity  patterns.  Even  if  the  background  density  or 
regularity  patterns  are  modified,  the  viewer  will  still  be  able  to  detect  target 
objects  by  height  preattentively. 


Texture 
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Figure  3:  Different  components  of  texture:  height,  density  and  regularity. 

It  is  recommended  to  use  color  for  rapid  target  detection  (~50  and  100  ms 
detection).  Maximum  of  7  isoluminant  (same  luminance  or  intensity  of  light) 
colors  can  be  displayed  simultaneously  while  still  allowing  for  the  rapid  and 
accurate  identification  of  any  one  of  the  seven  (Healey  1993).  Suggested  colors 
are  green,  yellow,  orange,  red,  and  purple. 
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Pre-attentive 

Task 

Pre-attentive 

Feature 

Guidelines 

To  enhance 
the  perception 
of  a  visual 
object’s  3D 
shape. 

Texture 

It  is  recommended  to  render  a  three  dimensional  (3D)  image  using  line  integral 
convolution  (see  Figure  4d)  which  will  give  the  best  perception  of  3D  shape, 
instead  of  Phong  shading  (see  Figure  4a).  The  other  methods  are  rendering  the 
3D  image  using  one  principal  direction  and  two  principal  directions,  as  shown 
in  Figure  4b  and  Figure  4c  respectively  (Kosara  et  al.,  2003). 

Figure  4:  Various  3D  rendering  that  can  be  used  to  enhance  the  surface 
perception  of  a  visual  object. 

To  track  a 
particular 
region 

Direction  of 
Motion 

Coherent  motion  can  be  used  to  separate  elements  into  coherent  groups 
(Nakayama  and  Silverman  1986).  In  other  words,  if  a  particular  group  moves 
in  the  same  direction,  the  individual  will  be  able  to  preattentively  detect  its 
motion  and  thus  the  region.  Even  though  the  objects  have  different  orientation, 
when  they  move  in  the  same  direction,  the  individual  will  be  able  to  detect  the 
region  preattentively  (seeFigure  5).  Oscillation  of  target  objects  must  also  be  in 
phase/coherent  (Driver,  et  al,  1992). 

^r~[ □  Direction  of  motion  ^r~i  □ 

. ► 

□  □ 

Figure  5:  When  the  visual  object  moves  in  the  same  direction,  orientation 
will  not  affect  the  user  in  detecting  the  region  preattentively. 

Color  and 
Texture 

Color  and  texture  can  be  combined  in  a  single  display  only  in  cases  where  the 
texture  targets  have  a  strong  perceptual  salience,  i.e.,  they  are  taller  and  denser 
(Snowden  1998).  It  is  recommended  that  color  is  used  to  represent  primary 
data  while  texture  is  used  to  represent  secondary  data.  This  is  because  color 
variation  interferes  with  an  observer’s  ability  to  see  texture  regions  based  on 
height  or  density,  but  variation  in  texture  has  no  effect  on  region  detection  by 
color. 
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Pre-attentive 

Task 

Pre-attentive 

Feature 

Guidelines 

Color  and 

It  is  recommended  that  color  is  used  to  represent  the  primary  data,  and  form  to 
represent  the  secondary  data.  This  is  because  visual  system  assigns  a  higher 
importance  to  color  than  to  form  during  boundary  detection.  Thus,  a  random 
color  interferes  with  form  boundary  detection,  but  a  random  form  has  no  effect 
on  color  boundary  detection  (Callaghan  1989). 

Form 

If  color  and  form  are  used  for  real-time  visualization,  the  sequence  of  frames 
should  be  displayed  at  10  frames  per  second  for  accurate  boundary  detection. 

For  accurate  and  rapid  detection  of  form,  color  must  be  held  constant. 
Otherwise,  color  would  be  a  better  feature  to  be  used  for  boundary  detection. 

It  is  recommended  that  intensity  is  used  to  represent  the  primary  data  and  color 
to  represent  the  secondary  data.  This  is  because  visual  systems  assign  a  higher 
importance  to  intensity  than  to  color  during  boundary  detection.  Thus,  a 
random  intensity  interferes  with  hue  boundary  detection,  but  a  random  hue  has 
no  effect  on  intensity  boundary  detection  (Callaghan  1984). 

(a)  tbl 

#  ## 

To  detect 
boundary 

Color  and 
Intensity 

•tut  .siti 

Figure  6:  Even  though  color  is  random  in  (a),  boundary  can  still  be 
detected  preattentively  because  the  visual  system  assigns  a  higher 
importance  to  intensity.  In  (b),  when  the  intensity  is  random,  it  becomes 
more  difficult  to  detect  the  boundary  using  color. 

When  two  independent  data  values  are  shown  on  a  single  display,  they  should 
be  represented  by  color  and  orientation.  Color  and  orientation  will  not  cause 
any  visual  interference  unlike  its  counterparts,  i.e.,  color  and  intensity  or  color 
and  form  (Healey,  et  al,  1993).  Boundary  can  be  detected  preattentively  either 
by  orientation  or  color  as  shown  in  Figure  7. 

Color  and 
Orientation 

#  #  ■  ■ 
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#####■■■  —  ■ 

#####■■■■■■■ 

####■■■■■■■■ 
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#####■--•—■ 

####♦##----- 

#########■■■ 

#*#########■ 

Figure  7:  Orientation  and  color  can  be  used  simultaneously  to  represent 
data  preattentively  without  affecting  each  other. 
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Pre-attentive 

Task 

Pre-attentive 

Feature 

Guidelines 

Any  given  texture  can  be  perceived  to  be  different  depending  on  the 
background,  thus  making  texture  contrast  the  strongest  visual  cue  for  texture 
segmentation. 

The  orientations  of  differing  textures  should  differ  by  at  least  30°  (Blake  and 
Holopigan,  1985).  However,  orientation  is  limited  and  should  not  be  used  as  a 
primary  means  for  texture  segregation. 

The  difference  in  texture  frequency  should  be  at  least  3  or  4  times  (Wilson  and 
Bergen,  1979).  E.g.  assuming  2  layers  of  textures  with  frequencies  fj  and  f2, 
either  f  >  f2,  or  f2  >  3fj  will  suffice  for  the  different  textures  to  be  segmented 
by  the  human  visual  system.  See  Figure  8  below  for  several  examples. 

To  identify 
segmentation 
of  regions 

T  exture 

kmHNi 

m  VtlHlSf 

'  ^  '  '  '  '  '  '  '/ 

'■  .<  y.  ,  ■  s,  >y."  ysy  ' 

Figure  8:  Top:  Only  texture  orientation  is  altered,  weakly  distinguishing 
the  word  TEXTURE.  Middle:  Texture  orientation  and  size  are  altered, 
strongly  distinguishing  the  word.  Bottom:  Texture  contrast  is  altered, 
strongly  distinguishing  the  word  (Ware  2000). 
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Appendix  B:  Gestalt  principles. 


Gestalt  Principle 

Example  Figure 

Verbal  Descriptor 

Law  of  Simplicity 

Every  object  is  perceived  in  a  way  that  the  resulting  structure  is 
as  simple  as  possible. 

Law  of  Closure 

I’"'. 

•  -  .  O' 

Tendency  to  close  gaps  and  complete  unfinished  objects. 

Law  of  Similarity 

□°o° 
n  °°0 

□no 

Elements  which  look  similar  (example,  size,  color,  orientation, 
velocity  and  shape)  are  perceptually  grouped  together  as  a 
object 

Law  of  Good  Continuity 

*...  >-  *  ' 
7^7"  ■■ 

Elements  that  are  smooth  and  continuous  are  perceptually 
grouped  together  than  ones  that  contain  abrupt  changes  in 
direction. 

Law  of  Connectedness 

A=DD 

□ 

Elements  that  are  physically  connected  are  perceptually 
grouped  together  as  a  object. 

Law  of  Proximity 

■  D 

□ 

□ 

m 

■■■ 

Elements  that  are  close  together  are  perceptually  grouped 
together  as  a  object 

Law  of  Common 
Fate/Common  orientation 

,i/i" 

'if  i 1 

Elements  with  the  same  moving  direction  or  orientation  are 
perceptually  grouped  together  as  a  object 

Law  of 

Balance/Symmetry 

«  » 

Elements  in  symmetrical  alignment  are  perceptually  grouped 
as  a  object 

Law  of  Common  Region 

^ 

Elements  tend  to  be  group  if  they  are  located  within  a  common 
region.  The  closed  contour  tends  to  be  perceived  as  the 
boundary  of  the  object. 
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Appendix  C:  Guidelines  for  the  use  of  depth  cues. 


Category 


Depth  Cue 


_ Guidelines _ 

Linear  perspective  is  widely  regarded  as  one  of  the  most  effective  sources  of  depth 
information  (Hone  and  Davies,  1995).  Figure  9  illustrates  an  example  of  linear 
perspective. 


Figure  9:  Linear  perspective  as  one  of  the  most  effective  depth  cue. 


Monocular  Perspective 
Static  Based  Cues 


Converging  parallel  lines  should  be  used  to  give  an  impression  of  increasing 
distance. 

Similar  objects  should  appear  smaller  with  increasing  distance. 

Texture  elements  of  uniformly  textured  surfaces  should  appear  smaller  with 
increasing  distance. 

Linear  perspective  should  be  avoided  when  there  is  a  need  to  convey  precise  values 
in  volumes  and  area  (Kosslyn  et  al.,  1983). 

Occlusion  of  objects  can  be  used  to  provide  binary  depth  information  between 
objects.  Objects  that  are  further  away  should  be  occluded  by  objects  that  are  nearer 
to  the  picture  plane  (see  Figure  10). 


Figure  10:  Depth  perception  based  on  occlusion,  (a)  The  larger  rectangle  is 
perceived  to  be  nearer,  (b)  The  smaller  rectangle  is  perceived  to  be  nearer. 
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Guidelines 


Category 


Depth  Cue 


Focus  effects  should  be  used  to  separate  foreground  objects  from  background 
objects  (Ware  2000).  Nearby  and  distant  objects  should  be  blurred  while  only 
objects  within  the  focal  distance  should  be  sharp  (see  Figure  11). 


Monocular 

Static 


Non 

Perspective 
Based  Cues 


Cast  shadows  act  as  a  depth  cue  to  locate  an  object  with  respect  to  some  surface  in 
an  environment  when  the  distance  between  the  object  and  the  surface  is  small 
(Madison  et  al.,  2001)  and  can  be  used  when  the  surface  geometry  is  unfamiliar  or 
complex  (Flaber,  1988).  Figure  12  below  illustrates  this  effect,  where  the  object  on 
the  right  appears  to  be  nearer  to  the  picture  plane  as  compared  to  the  object  on  the 
left. 


Figure  12:  Similar  objects  perceived  to  be  at  different  depths  due  to  cast  shadows. 

•  Artificial  spatial  cues  such  as  dropping  lines  (see  Figure  13)  to  some  surface  are 
also  effective  in  providing  depth  information  (Kim  et  ah,  1991).  Such  artificial  cues 
are  functionally  similar  to  cast  shadows.  However,  they  are  interpreted  much  more 
readily  as  there  will  be  no  conflict  with  lighting  directions  and  humans  perceive 
lines  more  readily  than  other  patterns  (Chambers  et  ah,  1983). 

Figure  13:  Dropping  lines  to  a  ground  plane  as  an  effective  artificial  spatial  cue. 

The  left  circle  appears  to  be  behind  the  right  circle. 

•  Luminance  and  contrast  have  little  to  no  effect  in  depth  perception  and  therefore 

should  not  be  used  for  portraying  depth  information  (Hone  and  Davies,  1993). _ 
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Category 

Depth  Cue 

Guidelines 

Monocular 

Dynamic 

Depth  from 
Motion 

Depth  or  structure  from  motion  provides  visual  cues  (location,  velocity,  acceleration 
and  direction)  humans  pick  up  when  objects  move  through  an  environment  (Gibson 
1986). 

•  Motion  parallax  should  be  used  to  convey  depth  information  when  objects  move  in 
parallel  with  the  viewer.  Objects  nearer  the  viewing  plane  should  appear  to  travel  at 
higher  velocity,  whereas  objects  further  from  the  viewing  plane  should  appear  to 
move  at  lower  velocity  (Hochberg,  1986;  Wann  et  al.,  1995). 

•  Kinetic  depth  effect  can  be  used  to  create  a  perception  of  a  rigid  3D  object  when  the 
view  of  a  2D  object  is  changed  (e.g.  via  rotation  in  3D  Cartesian  space)  (Wallach 
and  O’Connell,  1953).  E.g.  when  an  irregular  line  drawn  on  a  2D  screen  space  is 
rotated  about  a  vertical  axis,  the  perception  of  a  rigid  wire  bent  into  a  3D  object  is 
perceived. 

Binocular 

Stereopsis 

Stereoscopic  depth  perception  is  the  process  that  brings  the  retinal  images  in  the  two 
eyes  to  form  one  single  image.  During  the  process,  the  brain  locates  the  similarities 
between  the  two  individual  images  formed  in  each  eye  and  then  combines  the  individual 
differences  to  form  the  combined  image  (Wheatstone,  1838). 

•  Stereopsis  diminishes  rapidly  with  increasing  distance,  thus  its  usage  should  be 
limited  to  objects  within  the  near  and  middle  viewing  fields  not  more  than  10 
meters  from  the  viewpoint  (Nagata,  1993). 

•  Binocular  vision  should  be  used  only  for  comparing  relative  depth  of  objects  and 
not  for  absolute  or  concise  measurements  (Gillam,  1995). 

•  Stereopsis  is  particularly  useful  in  providing  visual  cues  for  objects  in  motion  as 
humans  are  more  sensitive  to  binocular  disparity  when  objects  are  moving  (Yeh, 
1993). 
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Introduction 


*  Lots  to  cover  in  a  little  time 
—  Graphic  Design 

—  Psychology 
—  Our  approach 

*  Will  only  hit  the  major  points 

*  We  are  expanding  the  paper  into  a  web-based 
document  for  ease  of  use 
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Graphic  Design 


•  Layout 

—  Literally  the  aesthetic  of  the  display  design 

•  Typography 

-  Serif  versus  san  serif 
—  Font  sizes 

*  Note  display  type  is  an  issue 
—  Small  versus  large  screen 

*  Note  viewing  angle 

—  Also  note  display  medium 

*  Paper  has  great  resolution! 
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Graphic  Design 


•  Color 

—  Associations 


Color 

Associated  meanings 

Color 

Associated  meanings 

Red 

Stop,  fire,  hot,  danger 

Blue 

Cold,  water,  calm,  sky, 
neutrality 

Yellow 

Caution,  slow 

White 

Neutrality 

Green 

Go,  OK,  clear,  vegetation, 
safe 

Grey 

Neutrality 

—  Consistency  and  redundancy 
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Graphic  Design 


*  Data  graphics 

—  Data-ink  ratio  (Tufte,  1983) 


—  Small  multiples  (Baker  &  Bushell,  1995) 
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Visual  Perception 


*  Perceptual  organization 


Gestalt  Principle 

Example  Figure 

Verbal  Descriptor 

Law  of  Simplicity 

Every  object  is  perceived  in  a  way  that  the  resulting  structure 
is  as  simple  as  possible. 

Law  of  Closure 

I'"'. 

Tendency  to  close  gaps  and  complete  unfinished  objects. 

Law  of  Similarity 

□°o° 
n  D^° 

□no 

Elements  which  look  similar  (example,  size,  color,  orientation, 
velocity  and  shape)  are  perceptually  grouped  together  as  a 
object 

Law  of  Good  Continuity 

'^1 

Elements  that  are  smooth  and  continuous  are  perceptually 
grouped  together  than  ones  that  contain  abrupt  changes  in 
direction. 

Law  of  Connectedness 

/VD 

□ 

Elements  that  are  physically  connected  are  perceptually 
grouped  together  as  a  object. 

Law  of  Proximity 

■ 

■ 

□□ 

□□□ 

Elements  that  are  close  together  are  perceptually  grouped 
together  as  a  object 

Law  of  Common 
Fate/Common  orientation 

,i/i" 

W 1 

Elements  with  the  same  moving  direction  or  orientation  are 
perceptually  grouped  together  as  a  object 

Law  of 

Balance/Symmetry 

«  » 

Elements  in  symmetrical  alignment  are  perceptually  grouped 
as  a  object 

Law  of  Common  Region 

- - v 

Elements  tend  to  be  group  if  they  are  located  within  a  common 
region.  The  closed  contour  tends  to  be  perceived  as  the 
boundary  of  the  object. 

lllllllllllllj) 
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Visual  Perception 


•  Pre-attentive  processing 

—  Certain  information  “pops”  out 
—  Other  information  cannot  be  “ignored” 

•  Two  examples 
—  Stroop 

—  Visual  Search 
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k, 


Visual  Perception 


Stroop  task  (say  the  color  as  quickly  as  possible) 
—  Humans  can’t  not  read  the  color  name 


RED 


RED 


Visual  Perception 


*  Find  the  vertical  bar 
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Visual  Perception 


*  Find  the  diagonal  bar 
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Visual  Perception 


*  Theories  of  visual  attention 

—  Space-based  -  attention  distributed  over  space 
irrespective  of  objects 

*  “Spotlight”  theories  of  attention 

—  Object-based  -  attention  distributed  based  on 
objects  (e.g.,  Gestalt  laws) 

*  Resource  allocation  theories  of  attention 
—  Both  (see  Logan,  1996) 
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Visual  Perception 


_ 


*  Theories  of  visual  attention  cues 


—  Exogenous  -  automatic  response  to  visual  cue 


B 


B 


G 


G 
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C 


X 
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X 


F 


R 


F 


R 


M 


M 

* 
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Visual  Perception 


_ 


*  Theories  of  visual  attention  cues 


—  Endogenous  -  under  voluntary  control 


B 


B 
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Visual  Perception 


*  3D  and  depth  perception 

—  Monocular  versus  binocular 

*  Do  we  need  3D 

—  What  is  true  3D 
—  When  is  it  useful? 
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Memory 


*  Memory  and  displays? 

—  Sensory  memory 

*  Sperling  experiments 
—  Short  term  memory 

*  Miller’s  magic  number 
—  Long  term  memory 

*  Declarative  versus  procedural 
—  Prospective  memory 

*  Remembering  to  remember 
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*  How  do  we  evaluate  graphical  displays  we  create? 

—  What  are  the  types  of  experiments? 

—  What  questions  can  be  asked? 

*  e.g.,  Situation  awareness,  memory  question,  or 
questions  about  the  IVS 

*  Examples  of  visual  information  fall  under  low  and  high 
level  visual  tasks 
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*  Low  level  visual  tasks  -  Wehrend  and  Lewis  (1990) 
—  Locate 
—  Identify 
—  Distinguish 
—  Categorize 
—  Cluster 
—  Distribution 
—  Rank 

—  Compare  within  entities 
—  Compare  between  relations 
—  Associate 
—  Correlate 
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Information  visualization  system 

(IVS) 


*  High  level  visual  tasks  -  Zhou  and  Feiner’s  (1998) 


Visual  Implication 

Type 

Subtype 

Elemental  tasks 

Organization 

Visual  grouping 

Proximity 

Associate,  cluster,  locate 

Similarity 

Categorize,  cluster,  distinguish 

Continuity 

Associate,  locate,  reveal 

Closure 

Cluster,  locate,  outline 

Visual  attention 

Cluster,  distinguish,  emphasize,  locate 

Visual  sequence 

Emphasize,  identify,  rank 

Visual  composition 

Associate,  correlate,  identify,  reveal 

Signalling 

Structuring 

Tabulate,  plot,  structure,  trace,  map 

Encoding 

Label,  symbolize,  portray,  quantify 

Transformation 

Modification 

Emphasize,  generalize,  reveal 

Transition 

Switch 
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*  Don’t  apply  en  masse  -  use  as  needed 

*  What  if  one  is  building  from  the  ground  up? 

—  Not  done  perfectly  (we  all  have  biases) 

—  Can  test  by  component  parts  first 

—  Interdisciplinary  background  teams  essential 

*  So  how  does  one  test  the  “final  product” 

—  Again  test  components  first 

—  Test  in  small  settings  before  large 
—  Keep  abreast  of  testing  research  and  theories 
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